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MEANS  AND  VARIANCES  OF  STOCHASTIC  VECTOR  PRODUCTS 
WITH  APPLICATIONS  TO  RANDOM  LINEAR  MODELS 


by 

Gerald  G.  Brown 
Naval  Postgraduate  School 
Monterey,  California 

and 

Herbert  C.  Rutemlller 
California  State  University 
Fullerton,  California 

Many  mathematical  models  in  operations  research 
require  computation  of  products  of  vectors  whose  elements 
are  random  variables.  Unfortunately,  analytic  results  for 
functions  of  Interest  are  only  obtained  through  highly 
restrictive,  often  unrealistic,  choices  of  prior  densities 
for  the  vectors'  elements.  Often,  an  investigation  is 
performed  by  discretizing  the  random  variables  at  point- 
quantile  levels,  or  by  outright  simulation.  This  paper 
addresses  the  problem  of  characterizing  the  Inner  product 
of  two  stochastic  vectors  with  arbitrary  multivariate 
densities.  Expressions  for  means  and  variances  of  vector 
products  are  obtained,  and  used  to  make  Tchebychef f-type 
probability  statements  which  can  accommodate  and  exploit 
stochastic  dependence.  Included  are  applications  to  stochastic 
programming  models. 


1.  INTRODUCTION 


Mathematical  models  In  operations  research  often 
require  the  calculation  of  quantities  which  are  sums  of 
products  of  random  variables.  (For  example,  see  the  realistic 
view  of  stochastic  linear  programming  taken  by  Tlntner  [15]). 
Sums  of  products  of  random  variables  are  the  generic  structural 
elements  fundamental  to  linear  models  exhibiting  complex 
randomness . 

In  the  Interest  of  tractable  mathematics,  It  Is 
commonly  assumed  either  that  the  random  variables  may  be  treated 
as  constants,  or  replaced  by  their  expectations.  Even  where 
these  simplifying  assumptions  are  not  made,  severe  parametric 
assumptions,  e.g.,  normality,  pairwise  Independence,  etc.,  are 
usually  required  to  achieve  analytic  probability  statements 
concerning  the  function  of  Interest.  Other  approaches  use 
weighted  sums  of  uniform,  exponential  or  other  variates  In 
approximation  schemes,  or  use  characteristic  functions  of  com- 
ponent densities  to  carry  out  the  Indicated  transformation  and 
attempt  Inversion  of  the  result.  Finally,  Monte  Carlo  solu- 
tions to  this  problem  may  be  achieved,  but  are  of  limited  value 
due  to  the  parametric  specificity  and  high  cost  of  results. 

For  excellent  examples  of  these  methods  see  Jagannathan  [7]; 
King,  Sampson,  and  Simms  [8];  Miller  and  Wagner  [9]; 

Searle  [12];  and  Sengupta  [13,  14]. 
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This  paper  addresses  the  problem  of  determining  the 
mean  and  variance  of  sums  of  products  of  random  variables  in 
the  most  general  linear  case,  i.e.,  arbitrary  correlated  multi- 
variate densities  for  all  components  In  the  model.  The  results 
obtained  are  useful  for  making  non-parametrlc  Tchebycheff-type 
probability  statements.  Particular  examples  are  given  which 
use  these  probability  statements  as  chance  constraints  in 
stochastic  linear  programming. 

Simplifications  of  this  model  for  moments  of  weighted 
sums  of  random  variables  have  been  presented  by  several  authors. 
For  Instance,  the  highly  successful  heuristic  given  by  Graves 
and  Whlnston  [5]  for  the  quadratic  assignment  problem  Is  based 
on  such  a derivation. 

2.  NOTATION  AND  DEVELOPMENT 

Let  b and  c be  two  random  n-element  column  vectors.  We 
denote  the  expectation  of  these  vectors  as 


The  symbols  b and  c will  Indicate  the  deviation  of  each  random 
vector  from  Its  expectation.  The  variance-covariance  matrix 
will  be  signified  by 


while  the  symbol  E*  will  signify  this  matrix  with  b and  c 
1 nterchanged . 
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Let  q be  a random  scalar  variable,  defined  as  the  inner 
product  of  these  random  vectors: 

q = b'c.  (1  ) 

Expressions  for  the  mean,  E(q),  and  variance,  V ( q ) , in 
terms  of  the  moments  of  b and  c are  derived  in  the  appendix. 
They  are: 

E(q)  - u^uc  + Tr  Ifac  f (2) 

where  Tr  Zbc  indicates  the  trace,  or  sum  of  the  diagonal 
elements  of  Zbc; 

V ( q ) = u'E*m  + 2u^  E(cc'b)  + 2u^E(bb’c) 

+ E(B'cc'B) - ( T 1 * lhc)2  . (3) 

Equations  (2)  and  (3)  represent  the  most  general 
outcome,  where  any  element  of  b may  be  correlated  with  any 
other  element  of  b or  c.  Two  special  cases  occur  frequently. 
First,  suppose  that  the  elements  of  b are  correlated  with  one 
another,  the  elements  of  c are  correlated  with  one  another, 
but  that  b and  c are  stochastically  independent.  Then 


E ( q ) = ubuc  , (4) 

V(q)  ■ m(,  Ecc  rbb  uc  * TrUbb!Tcc)  . 


(5) 
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Secondly,  we  note  that,  If  all  elements  of  both  matrices 
are  stochastically  Independent,  I Is  a diagonal  matrix,  and 
computation  of  (5)  Is  greatly  simplified. 

An  assumption  frequently  made  Is  that  elements  of 
vectors  b and  c have  a multivariate  normal  distribution.  If  we 
adopt  this  parametric  condition,  then,  even  In  the  general  case 
where  b and  c are  correlated,  (3)  Is  greatly  simplified.  For 
multivariate  normal  distributions,  E(bb'c)  * E(cc'b)  * 0,  and 
all  higher  moments  are  expressible  In  terms  of  the  first  two 
moments  [1,  p.  38].  Equation  (3)  becomes 

V(q)  = u'Z*u  + TrUbb£cc)  + TrUbc)  . (6) 

3.  AN  ESTIMATION  EXAMPLE 

A job  shop  is  bidding  on  the  manufacture  of  units  which 
will  consume  two  raw  materials  and  skilled  labor.  Let  b be  the 
vector  of  material  and  labor  requirements  per  unit  of  product, 
and  c the  vector  of  respective  costs  per  unit  of  b.  Then  the 
total  manufacturing  cost  per  unit,  q,  may  be  computed  with 
equation  (1).  Due  to  unknown  factors  In  the  manufacturing 
process  (e.g.,  proportion  of  scrap  units  produced)  the  elements 
of  b are  random  variables.  Further,  although  the  skilled  labor 
cost  Is  fixed  by  contract,  other  elements  of  c are  random 
variables  dependent  upon  commodity  prices  and  other  factors. 

Historical  cost  accounting  data  are  available  for 
similar  products  In  the  form  of  job  orders  and  associated 
material  requisitions  and  labor  charges,  and  are  used  as  prior 


Information  to  determine  the  means  and  variance-covariance 
matrices  of  b and  c as  follows: 
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4.2  units  of  I 

6.8  units  of  II 

4.8  hours  of  labor; 


bb 


cc 


$2.15  per  uni t of  I 
$1.50  per  uni t of  1 1 


$6.20 

per 

hour 

0.7 

0.8 

0.2 

0.8 

1.2 

0.3 

C\J 

o 

0.3 

0.5 

0.6 

0.1 

0.0 

0.1 

0.2 

0.0 

0.0 

0.0 

0.0 

be 


Prices  of  the  two  raw  materials  are  positively  corre- 
lated, as  are  the  amounts  consumed  and  the  labor  expended.  In 
this  example,  there  Is  no  correlation  between  commodity  prices 
and  quantities  consumed  during  manufacture,  so  (4)  and  (5)  are 
the  appropriate  equations. 
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E(q)  = <uc 

= $48.99 

v(q)  ■ u;ibbuc  + u^ccub  + Tr(ZbbEcc) 

= 41.23  + 25.54  + 0.82 
= 67.59  • 

Hence,  the  expected  cost  for  one  completed  unit  Is  $48.99  with 
a standard  deviation  of  $8.22. 

The  (unknown)  probability  distribution  of  q depends  upon 
the  mathematical  form  of  the  multivariate  density  functions  of 
b and  c,  and  the  transformation,  equation  (1).  However,  the 
moments  calculated  above  are  exact,  and  we  may  use  them  to  make 
non-parametrl c statements  about  the  distribution  of  q.  Two 
Illustrations  follow. 

Tchebychef f 1 s Inequality 

Regardless  of  the  underlying  distributions,  at  least 

2 

1 - 1/k  of  the  observations  of  q will  be  within  k standard 
deviations  of  E(q);  e.g.,  there  is  a 75  percent  probability 
that  the  price  will  lie  in  the  Interval  $32.55-65.43. 

Camp-Meldell  Inequality 

Some  mild  assumptions  about  the  distribution  of  q lead 
to  much  tighter  bounds  [2].  If  we  assume  that  the  distribution 
Is  unlmodal,  with  the  mode  occurring  at  E(q),  then  at  least 
1 - 1/2. 2 5 k ^ of  the  observations  will  lie  within  k standard 
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deviations  of  the  mean;  e.g.,  there  Is  an  89  percent  probability 
that  the  price  will  lie  In  the  Interval  $32.55-65.43. 

For  more  general  treatments  of  Tchebychef f- type  bounds, 
see  [4]  and  [9]. 

4.  LINEAR  PROGRAMMING  APPLICATIONS 

The  chance  constrained  programming  model  of  Charnes 
and  Cooper  [3]  presents  Interesting  applications  of  these 
techniques.  A deterministic  linear  program  may  be  stated 


Max 

C ' X 

s.t. 

Ax  < b » 

(7) 

with  A an  mxn  constant  matrix  of  technological  coefficients, 
c a vector  of  constant  prices  and  b a vector  of  constant 
resource  levels.  If  we  change  the  sense  of  b to  that  of  a 
vector  of  resource  levels  realized  from  a multivariate  density 
function,  f(b),  then  a reformulation  of  (7)  yields 

Max  c'x 

s.t.  Pr[Ax  _<  b]  _>  a » (8) 

where  a Is  a set  of  probabilities  of  individually  satisfying 
each  of  the  m constraints.  This  problem  may  be  solved  by  using 
the  density  function  asserted  for  b to  find  a set  of  bounds, 

8 (a,f  (b)j  , such  that 


Pr[b  > 8]  > 


a , 


8 


and.  If  f(b)  Is  free  of  terms  Involving  x,  reformulating  (8) 
to  the  "deterministic  equivalent"  form 

Max  c'x 


s . t.  Ax  < 8 . 


(9) 


Alternatively,  let  a be  the  scalar  joint  probability  that 
solutions  of  (9)  will  ultimately  prove  simultaneously  feasible 
for  all  constraints.  If  the  variates  b are  independent, 
m 

a * II  a.,  but  If  the  terms  In  b are  correlated, 

J-l  J 

extraction  of  a from  f(b)  may,  or  may  not,  be  possible  or 
practical  depending  on  the  mathematical  form  of  f(b),  the 
dimensionality  of  b,  and  so  forth. 

If  f(b)  Is  unknown  or  mathematically  uncooperative  we 
may  use  the  moments  of  the  vector  b v.o  specify  8 nonparametr  1 c- 
ally  with  Tchebychef f 1 s inequalities.  For  Instance 


pr[bj 


uj  - yy  ■ y 


j s 1 , 2 , . . . ,m 


Also,  a frequent  and  reasonable  simplification  is 

k,  = . . . ■ k * k . 

I m 

Further,  we  can  specify  a bound  for  a by  using  a 
multivariate  Tchebycheff  inequality. 


Pr[bj  ■=  “j  • yy  * y 


,m] 


1 


m 1 

a < E — ; 

" J*'  kj 


(10) 


or,  for  k 


• • 


k 
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Pr[b  < y - k ( (dlag  I))1/2  « B]  = 1 - a < ^ . (11) 

1/0  

Here,  (dlag  Z)  ' denotes  a column  vector  with  components  /o.,  . 

J J 

Another  Tchebycheff  Inequality  presented  by  Olkin  and 
Pratt  [11]  can  be  used  to  specify  a bound  for  a.  If  we  use  R 
to  represent  the  correlation  matrix  of  b so  that 


n 


jk 


°Jk 

/ajjckk 


and  simplify  the  presentation  by  choosing  the  constants 

k,  * ...  * k„  * k , then 
l m 

Pr[b  < y - k ( ( d 1 ag  Z))^Z  = B]  s 1 - a 

-,2 

< -l--g  |/p  + /(mZ  - p ) (m  - 1 ) 

“ k*m L J 

wl  th 

p * e'lle,  where  e Is  a vector  of  l's. 


(12) 


5.  A LINEAR  PROGRAMMING  EXAMPLE 
To  Illustrate,  using  b from  the  example  In  section  3, 


L0. 34  0. 


1 I = 6.20  ; 


1 - a < ^-4^  by  use  of  (12) 
“ k* 


10 


2 

e note  that  this  Is  a better  bound  than  the  3/k  produced  by 

11). 

or  k * 3 we  find 

a ^ 0 . 7 1 ; 

/ 4 . 2 - 3 ( .84) 

6 = u - k ( d 1 a g Z)1/2  = ( 6.8  - 3(1.10) 

\ 4.8  - 3 ( .71 ) 

If  (7)  is  further  generalized  to  allow  both  b and  c to  be  random 
vectors,  possibly  correlated,  we  can  easily  incorporate  the  expec- 
tation of  the  objection  function  in  a deterministic  equivalent. 
Alternately,  following  the  development  of  [3] 
x = Db  , 

with  D an  nxm  matrix,  we  reformulate  (7)  to 
Max  E [c ' x ] 

s . t.  Pr[Ax  <_  b]  _>  a 

x = Db  , (13) 

which  by  substitution  becomes 
Max  E[c ' Db] 

S.  t.  Pr[ADb  < b]  > a . 

Using  the  notation  and  results  of  section  2,  we  derive  for  the 
objective  function: 


E [c  ' Db]  * v'c  Dpb  + Tr{Dlbc)  . 
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The  constraint  set  may  be  restated 

Pr[ADb  < b]  = Pr[ADb  - b _<  0] 

* Pr[(AD  - I )b  £ 0]  _>  a . 

Thus,  using  the  moments  of  b It  follows  [1,  p.  24]  that  the 
vectors  of  expectations  and  variances  are 

E[ { AD  - I )b]  = (AD  - I)ub  * 5(D)  ; 

V [ ( AD  - I )b]  = dlag  {(AD  - I)Zbb  (AD  - I)'} 

s y(D)  • 

The  procedure,  then,  Is  as  follows: 

(1)  Specify  an  acceptable  o^,  the  probability  of  feasibility 
for  each  constraint. 

(2)  Use  (10),  (11),  or  (12)  to  generate  the  set  of  k's  and  a. 

A set  of  slack  variables,  v,  Is  Introduced.  Adopting  the  symbol 
to  represent  congruent  multiplication  of  two  commensurate 
matrices  [6,  p.  549  ff]  , the  deterministic  equivalent  program- 
ming model  for  (13)  becomes 

Max  ucDyb  + ^bc  * 

s.t.  v - 6 (D ) <_  0 

v*v  + k*k*{  <5  (D)*6  (D)  - y(D)*y(D)}  < 0 


v > 0 . 


(14) 
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Finally,  a solution,  0*,  to  this  problem  can  be  used  to  make 
nonparametric  interval  estimates  for  the  value  of  the  optimal 
objective  function  using  equation  (3)  or  simplifications  (5)  or 
(6),  much  as  In  the  numerical  example  in  the  preceding  section. 

This  variance  can  be  used  as  the  objective  function  In 
another  model  proposed  by  [3] 

Min  V[c  ' x J 

s . t.  Pr[Ax  <_  b]  >_  a 

x = Db  • 

and  Its  deterministic  equivalent  which  follows  the  development 
of  (14). 

A "partitioned"  stochastic  programming  model  considers 
the  problem 

Max  c ' x + d * y 

s . t . Ax  + By  _<  b ; 

where  y Is  a set  of  external,  uncontrollable  prior  random 
variables,  d a vector  of  random  prices,  and  B a random 
technological  coefficient  matrix.  By  using  the  moments  of 
y,  d,  B,  and  b,  the  problem  may  be  restated 

Max  c ' x + Cq 

s . t.  Ax  l^o’ 
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with  the  moments  of  bg  and  Cg  specified  exactly,  and  solved 
as  previously  shown. 

6.  CONCLUSION 

Estimation  of  stochastic  vector,  or  matrix,  ptoducts 
may  be  virtually  Impossible  by  use  of  multivariate  density 
functions  and  statistical  transformations  without  severe  and 
unrealistic  simplifying  assumptions  about  the  random  variates 
Involved.  Simulation  can  lead  to  approximations  for  a given 
problem,  but  at  high  computation  cost  and  specificity  for 
resul ts . 

However,  an  approach  using  multivariate  moments,  which 
are  available  from  either  empirical  data  or  analytic  densities, 
leads  to  exact  statements  about  the  mean  and  variance  of  the 
vector  products  and  useful  Tchebyc hef f- type  probability 
estimates.  These  estimates  are  very  helpful  in  generalizing 
the  mathematical  models  of  operations  research. 
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APPENDIX 

Derivation  of  Expressions  for  the  Mean  and  Variance 
of  the  Inner  Product  of  Two  Vectors 

q = b 1 c ; 

q = (b  + ub) 1 (c  + uc) ; 
q = b'c  + 2 b * uc  + 2p^c  + ■ 


Noting  that  E(b)  = E(c)  = 0 ; 

E(q)  = E(b'c)  + ybPc  ; 

E(q)  ■ I.^c  + Tr  Ebc  ; 

V(q)  = E(q2)  - lE(q)]2  ; 

V(q)  ■ E(b'c)2  - (m^mc)2  - 2m^c  Tr  Zbc  - (Tr  Eb(.)2  . 

In  terms  of  moments  about  the  mean, 

E(b*c)2  = E [ ( b + Mb)'(c  + pc)]2  ; 

E ( b 1 c ) 2 = E[b'cc'b  + u ^ c c * p b + p^.bb'pc  + (pbMc)2  + 2pbcc'b 

+ 2n^bb'c  + 2y^Mcb’ c + 2ubcb'uc+  2p^bpbuc 

+ ^b^b^1- 

We  note  that 

E(cc’)  - >:cc  ; 

E(bb')  = ; 

E(b'c)  = Tr  Jbc  ; 

E(b)  « E (c)  * 0 . 


Hence , 


E (b ' c ) 2 


We  have 


1 7 


= E(b-cc-b)  ♦ >^ccMb+  »'cZbhvc+  2lJb}:cbMC  + (,1bMc)  + 2^bE(cc'b) 
+ 2u*E(bb'c)  + 2u‘mc  Tr  >:bc  . 


m 1 £ * p = p ! Y.  p . + p'L  , u + 2 p |!  .M  ; 

K * b cc  b c bb  c b cb  c 


V(q)  = E(b'cc'b)  + u'>:*u  + 2p^E(bb'c)  + 2p^  E(cc'b) 


- <Tr  : be’ 
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